Notebook to test basic functionalities¶
Imports¶
In [1]:
# Settings for notebook visualization
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = 'all'
%matplotlib inline
from IPython.core.display import HTML
HTML("""<style>.output_png img {display: block;margin-left: auto;margin-right: auto;text-align: center;vertical-align: middle;} </style>""")
Out[1]:
In [2]:
# Necessary imports
import os
import numpy as np
import pandas as pd
import matplotlib as plt
import quantstats as qs
from datetime import datetime, timedelta
print("Libraries imported correctly")
In [3]:
os.chdir("/Users/Sergio/Documents/Master_QF/Thesis/Code/Algorithmic Strategies")
%run Functions.ipynb
Data¶
In [4]:
ini_equity_default = 100
commision_default = 2/130000 + 12.5/130000 #0.000111538462, around 0.011..% of the equity
In [5]:
%run Functions.ipynb
#data = get_sp500_data(from_local_file=False, save_to_file=True)
data = get_sp500_data(from_local_file=True, save_to_file=False)
data = data[['Open', 'Close']]
data['Market_daily_ret'] = data['Close'].pct_change().fillna((data['Close']-data['Open'])/data['Open'])
data = data.loc['2000':'2020', ['Close', 'Market_daily_ret']]
data.info()
data.tail()
data['Close'].plot(title='SP500', legend=True)
Out[5]:
Out[5]:
Simple backtests with and without transaction costs¶
The purpose of this section is to check backtests based on Moving Average crossovers. There are 3 subsections:
- Signals on SP500. This will allow to see how MA crossovers produce buy/sell signals when different MA crossovers occur.
- Backtest without transaction costs: It allows to see how buy/sell signals affect returns from the strategy.
- Backtest with transaction costs: It allows to see the effect of transaction costs on returns from the strategy.
These are two backtests from 2018-01-01 to 2019-12-31. This first one is done without transaction costs, and the second one adds a transaction cost every time the strategy changes the position.
Using the MAs of 75 and 200 periods, there are two crosses. On 2018-12-26 and 2019-04-24. 'Strat_position' changing at the Close of that day will cause the strategy to switch the position on the market. This change is added at the end of such days, when we would send the change of position to the broker.
In order words, if this strategy was live trading, it uses the price a few minutes before the end of the trading day as the Close price, to take the decision about whether to change the position or not. This change in position has transaction costs, which affects the returns of such days.
In [6]:
date_fmt = '%Y-%m-%d'
first_day = datetime.strptime('2018-01-01', date_fmt)
last_day = datetime.strptime('2019-12-31', date_fmt)
fast_ma = 75
slow_ma = 200
# fast_ma = 25 # Change to this parameters to see more changes in Strat_position
# slow_ma = 40
Signals on SP500¶
In [7]:
%run Functions.ipynb
df = data.loc[first_day:last_day].copy()
plot_sp500_with_ma_signals(df, fast_ma=fast_ma, slow_ma=slow_ma)
df.head(3)
df['2019-04-20':'2019-04-25']
Out[7]:
Out[7]:
Backtest without transaction costs¶
This allows to test a moving average crossover strategy. Useful to see how a backtest is done and columns from the final dataframe after doing the backtest:
- Close: Close price of the benchmark.
- Market_daily_ret: daily percentage returns of the benchmark. Calculated from the previous Close price to the actual Close price.
- Strat_position: This is the most important column from the backtest. Daily returns, cummulative returns and costs are calculated after the strategy position is defined. Refers to the position that we have AT THE END of such day. If the Strat_position changes from the previous day, we add transaction costs to the strategy. Calculated based on moving averages.
- Strat_daily_ret: daily percentage returns of the strategy. Calculated from the previous Close price to the actual Close price.
Strat_daily_ret = Market_daily_ret Strat_position.shift() (1-commission)
Note: commission is paid if Strat_position != Strat_position.shift() - Costs: Daily transaction costs in USD - . (This is the trickiest part of the whole framework. It´s easier to see the final dataframe to check that it works). Formula:
Costs (in USD) = Strat_cum_ret.shift() [1+[Market_Daily_retStrat_position.shift()]] * (commission)
- Long_only: Refers to open a Buy&Hold position. This is useful for the walk-formard optimisization. It is set to 1 when B&H strategy performed better than all tested parameters In Sample.
- Market_cum_ret: Market cummulative returns. This is proportional to the Close price of the benchmark, starting at a price of initial_equity
If initial_equity is set to the initial price of the benchmark, Market_cum_ret = Close
- Strat_cum_ret: Strategy cummulative return. Built from 'Strat_daily_ret' and starting with a price of initial_equity.
Strat_cum_ret = Strat_cum_ret.shift() [1+[Market_Daily_retStrat_position.shift()]] * (1-commission)
In [8]:
loc = data.index.get_loc(df.index[0])
ini_money = data.iloc[loc - 1]['Close'] # We assume we can buy at the Close of the previous day to the start of the backtest
#ini_money = 100
previous_position = 1
fast_ma = 75
slow_ma = 200
strategy = ma_crossover(df, fast_ma, slow_ma)
In [9]:
%run Functions.ipynb
df = data[first_day:last_day].copy()
print("Backtest without transaction costs: \nini_money = {:.2f}".format(ini_money))
df = backtest_print_plot(df, strategy, strat_params=(fast_ma, slow_ma),
previous_position=previous_position,
ini_equity=ini_money, commision=0, with_legend=True)
print("\nFirst three days: \n\tWe can see how we had entered the market before the first day (because previous_position=1).\n\t"
"And since costs = 0:\n\t"
"On 2018-01-03, Strat_cum_ret = 2695.810059 * [1+[0.006399*1.0]] = 2713.060059")
display(df.head(3))
print("\nAt the end of 2018-12-26 there is a change of Strat_position from 1.0 to 0.0: \n\t"
"And since costs = 0:\n\t"
"On 2018-12-26, Strat_cum_ret = 2351.100098 * [1+[0.049594*1.0]] * [1-Costs] = 2467.699951 (Closing the position didn´t cost us anything)\n\t"
"On 2018-12-27, Strat_cum_ret = 2467.699951 * [1+[0.008563*0.0]] = 2467.699951")
display(df.loc['2018-12-21':'2018-12-27']) # Change of Strat_position from 1 to 0
print("\nAt the end of 2019-04-24 there is a change of Strat_position from 0.0 to 1.0: \n\t"
"And since costs = 0:\n\t"
"On 2019-04-24, Strat_cum_ret = 2467.699951 * [1+[-0.002192*0.0]] * [1-Costs] = 2467.699951 (Opening a position didn´t cost us anything)\n\t"
"On 2019-04-25, Strat_cum_ret = 2467.699951 * [1+[-0.000369*1.0]] = 2466.789435")
display(df.loc['2019-04-22':'2019-04-25']) # Change of Strat_position from 0 to 1
plt.show()
df['Costs'].plot(title='Transaction costs in USD ($)', figsize=(12,1), color='r');
With transaction costs¶
We test how a transaction costs on the backtest affects 'Costs', and therefore, 'Strat_daily_ret' and 'Strat_cum_ret'. We can see change in 'Strat_position' on 2018-12-26 and 2019-04-24.
We can also see how previous_position has an effect on the 'Costs' of the first day. previous_position refers to the position that we had in the market on the previous day to the start of the strategy. This parameter is set to 0 by default, and it will be useful for the walk-forward optimization, in which we may end a period_t adn strat a period_t+1 being long on the market, which would safe us transaction costs from selling and buying.
A very high commision of 0.50 (50% of the actual equity for entering a position) shows well how transaction costs work in the backtest. Two aspects are worth being checked in this specific test:
- We can see how we loose 50% of the capital on the days that we change the position on the market (2018-12-26 and 2019-04-24).
- The only difference between first and second backtest is the _previousposition parameter.
- In the first backtest we are initially with previous_position=0, which means that we are out of the market. Since our strategy tells us to enter the market from the first day, we have to pay a transaction costs of 50% of our capital, and we can see that reflected on 'Costs', 'Strat_daily_ret', and 'Strat_cum_ret'.
- In the second backtest we are initially with previous_position=1, which means that we have a buy position in the market. Since our strategy tells us to keep long in the market from the first day, we don´t have to pay a transaction costs. We can see the difference with the previous case on 'Costs', 'Strat_daily_ret', and 'Strat_cum_ret'.
In [10]:
loc = data.index.get_loc(df.index[0])
#ini_money = data.iloc[loc - 1]['Close'] # We assume we can buy at the Close of the previous day to the start of the backtest
ini_money = 100 # So its easier to calculate transaction costs
commision = 0.5
#previous_position = 1
fast_ma = 75
slow_ma = 200
strategy = ma_crossover(df, fast_ma, slow_ma)
In [11]:
%run Functions.ipynb
df = data[first_day:last_day].copy()
print("Backtest with transaction costs and previous position in the market = 0: \nini_money = {:.2f}".format(ini_money))
print("\t\tStrat_cum_ret = Strat_cum_ret.shift() * [1+[Market_Daily_ret*Strat_position.shift()]] * (1-commission)")
df = backtest_print_plot(df, strategy, strat_params=(fast_ma, slow_ma),
previous_position=0, ini_equity=ini_money, commision=commision,
figsize=(12,3), with_legend=True)
display(df.head(3))
plt.show()
_ = df['Costs'].plot(title='Daily transaction costs ($)', figsize=(12,2), color='r')
plt.show()
df[['Market_daily_ret', 'Strat_daily_ret']].plot(figsize=(12,3))
plt.show()
print("Backtest with transaction costs and previous position in the market = 1: \nini_money = {:.2f}".format(ini_money))
df = backtest_print_plot(df, strategy, strat_params=(fast_ma, slow_ma),
previous_position=1, ini_equity=ini_money, commision=commision,
figsize=(12,3), with_legend=True)
display(df.head(3))
plt.show()
_ = df['Costs'].plot(title='Daily transaction costs ($)', figsize=(12,2), color='r')
plt.show()
df[['Market_daily_ret', 'Strat_daily_ret']].plot(figsize=(12,3));
#df.iloc[[0,1,2,-2,-1]]
# df.loc['2018-12-21':'2018-12-27']
# df.loc['2019-04-22':'2019-04-25']
One more example with transaction costs¶
More realistic commision, but still big (and visible in the plot): 1%
In [12]:
loc = data.index.get_loc(df.index[0])
ini_money = data.iloc[loc - 1]['Close'] # We assume we can buy at the Close of the previous day to the start of the backtest
#ini_money = 100
commision = 0.01
previous_position = 1
strategy = ma_crossover(df, fast_ma, slow_ma)
In [13]:
%run Functions.ipynb
df = data[first_day:last_day].copy()
print("Backtest with transaction costs: \nini_money = {:.2f}".format(ini_money))
print("\t\tStrat_cum_ret = Strat_cum_ret.shift() * [1+[Market_Daily_ret*Strat_position.shift()]] * (1-commission)")
df = backtest_print_plot(df, strategy, strat_params=(fast_ma, slow_ma),
previous_position=previous_position, ini_equity=ini_money, commision=commision,
figsize=(12,3), with_legend=True)
print("\nFirst three days: \n\tWe can see how we had entered the market before the first day (because previous_position=1).\n\t"
"And since costs = 0:\n\t"
"On 2018-01-03, Strat_cum_ret = 2695.810059 * [1+[0.006399*1.0]] = 2713.060059")
display(df.head(3))
print("\nAt the end of 2018-12-26 there is a change of Strat_position from 1.0 to 0.0: \n\t"
"And since costs = 0:\n\t"
"On 2018-12-26, Strat_cum_ret = 2351.100098 * [1+[0.049594*1.0]] * [1-0.01] = 2443.022952 (Closing the position cost us 1% of the equity at the end of 2018-12-26)\n\t"
"\t\t\tCosts = 2351.100098 * [1+[0.049594*1.0]] * [0.01] = 24.677\n\t"
"On 2018-12-27, Strat_cum_ret = 2443.022952 * [1+[0.008563*0.0]] = 2443.022952")
display(df.loc['2018-12-21':'2018-12-27']) # Change of Strat_position from 1 to 0
print("\nAt the end of 2019-04-24 there is a change of Strat_position from 0.0 to 1.0: \n\t"
"And since commission = 0.01:\n\t"
"On 2019-04-24, Strat_cum_ret = 2467.699951 * [1+[-0.002192*0.0]] * [1-Costs] = 2443.022952 (Opening the position cost us 1% of the equity at the end of 2019-04-24)\n\t"
"On 2019-04-25, Strat_cum_ret = 2443.022952 * [1+[-0.000369*1.0]] = 2443.022952")
display(df.loc['2019-04-22':'2019-04-25']) # Change of Strat_position from 0 to 1
plt.show();
_ = df['Costs'].plot(title='Daily transaction costs ($)', figsize=(12,2), color='r');
#_ = df['Costs_in_pct'].mul(df['Strat_cum_ret']).plot(title='Daily transaction costs (%)', figsize=(12,2), color='r')
plt.show();
df[['Market_daily_ret', 'Strat_daily_ret']].plot(figsize=(12,3));
plt.show();
# df.iloc[[0,1,2,-2,-1]]
# df.loc['2018-12-21':'2018-12-27']
Performance metrics¶
In [14]:
metrics = calculate_performance_metrics(df, strat_name='MA Crossover')
metrics
Out[14]: